Light field image processing: an overview

Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data

X-ray transmission intelligent coal-gangue recognition method

The coal-gangue image recognition is an important part of coal-gangue separation technology based on pseudo dual energy X-ray transmission (XRT). However, it is difficult to segment the coal-gangue image due to the close proximity or occlusion of coal-gangue, and it is easy to cause classification and recognition errors of coal-gangue based on artificial threshold discrimination. Due to the above influence, existing coal-gangue recognition methods have low precision. In this paper, an X-ray transmission intelligent coal-gangue recognition method is proposed. A U-Net model combined with the receptive field block (RFB) is used to realize the effective segmentation of the pseudo dual energy X-ray coal-gangue image, which is termed as RFB + U-Net model. The problem that the recognition precision is affected by the close proximity or shielding of coal-gangue is solved. The recognition features of coal-gangue are the minimum gray value of the low-energy image in the gray level features of coal-gangue image, and the minimum value and the average difference of sharpened low-energy image in the texture features. A multi layer perceptron (MLP) model is used to realize coal-gangue recognition. Experimental results show that the RFB+U-Net model is superior to the active contour model, U-Net model and SegNet model in terms of coal-gangue segmentation accuracy, coal-gangue particle size precision, coal-gangue pixel mean intersection ratio and image segmentation effect. The reasoning time of the model is short, meeting the real-time requirements of coal-gangue image segmentation. When the number of hidden layers in the MLP model is 8, the average coal-gangue recognition accuracy under two test sets is more than 87%. Under the same data set and experimental conditions, the average recognition accuracy and gangue removal rate of the MLP model are higher than those based on Bayesian classifier, support vector machine, logic regression, decision tree, gradient boosting decision tree and K-nearest neighbor algorithm. The coal carrying rate of gangue shall not exceed 3%, meeting the requirements of actual dry coal-gangue separation

Disentangling Light Fields for Super-Resolution and Disparity Estimation

Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/.Comment: Published on IEEE TPAMI. Project page: https://yingqianwang.github.io/DistgLF

Gas transfer velocities of methane and carbon dioxide in a subtropical shallow pond

Two diel field campaigns under different weather patterns were carried out in the summer and autumn of 2013 to measure CO2 and CH4 fluxes and to probe the rates of gas exchange across the air–water interface in a subtropical eutrophic pond in China. Bubble emissions of CH4 accounted for 99.7 and 91.67% of the total CH4 emission measured at two sites in the summer; however, no bubble was observed in the autumn. The pond was supersaturated with CO2 and CH4 during the monitoring period, and the saturation ratios (i.e. observed concentration/equilibrium concentration) of CH4 were much higher than that of CO2. Although the concentration of dissolved CO2 in the surface water collected in the autumn was 1.24 times of that in the summer, the mean diffusive CO2 flux across the water–air interface measured in the summer is almost twice compared with that in the autumn. The mean concentration of dissolved CH4 in the surface water in the autumn was around half of that in the summer, but the mean diffusive CH4 flux in the summer is 4–5 times of that in the autumn. Our data showed that the variation in gas exchange rate was dominated by differences in weather patterns and primary production. Averaged k600-CO2 and k600-CH4 (the gas transfer velocity normalised to a Schmidt number of 600) were 0.65 and 0.55 cm/h in the autumn, and 2.83 and 1.64 cm/h in the summer, respectively. No statistically significant correlation was found between k600 and U10 (wind speed at 10 m height) in the summer at low wind speeds in clear weather. Diffusive gas fluxes increased during the nights, which resulted from the nighttime cooling effect of water surface and stronger turbulent mixing in the water column. The chemical enhancements for CO2 were estimated up to 1.94-fold in the hot and clear summer with low wind speeds, which might have been resulted from the increasing hydration reactions in water due to the high water temperature and active metabolism in planktonic algae. However, both the air and surface water temperatures decreased continually, and relatively lower temperature and overcast weather with occasionally light rain dominated the second campaign in the autumn. The concentration of dissolved oxygen in the surface water and U10 controlled gas transfer velocities of CO2 and CH4, respectively, in the cool autumn. When the surface water temperature was higher than the air temperature, higher CO2 flux was observed because the water body was unstable and overturned quickly, inducing quick CO2 emitted from plankton algae in surface water to the atmosphere